Tsallis entropy: How unique? 



Sumiyoshi Abe 

Institute of Physics, University ofTsukuba, Ibaraki 305-8571, Japan 

Abstract It is shown how, among a class of generalized entropies, the Tsallis entropy 
can uniquely be identified by the principles of thermodynamics, the concept of stability 
and the axiomatic foundation. 



1. Introduction 

The additivity requirement for the thermodynamic quantities puts constraint on 
symmetries of the phase/configuration space of the system under consideration: it is 
indivisibly connected with homogeneity of the system, which is a familiar additional 
assumption in ordinary thermodynamics . According to our experience, this requirement 
is not too stringent, and in fact it is fulfilled by ordinary thermodynamic systems. 

Today, thermodynamically exotic complex systems/processes are attracting great 
attention. Examples include colossal magnetoresistance manganites, amorphous and 
glassy nanoclusters and high-energy collision processes. A common feature in these is 
that such systems stay in nonequilibrium stationary states for significantly long periods 
(compared to typical time scale of their microscopic dynamics) with preserving scale 
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invariant and hierarchical structures. There, the phase/configuration spaces are 
generically inhomogeneous, and accordingly the naive additivity requirement may not be 
satisfied any more. 

In this article, we discuss Tsallis' nonadditive entropy and show how it can uniquely 
be identified by the principles of thermodynamics, the concept of stability and the 
axiomatic foundation. In Sec. 2, a general composition law for thermodynamic entropy 
compatible with the zeroth law of thermodynamics is discussed. There, three generalized 
entropies, the Tsallis entropy, the normalized Tsallis entropy and the Renyi entropy, are 
shown to be consistent with the zeroth law. In Sec. 3, the concept of stability is examined 
for these generalized entropies, and it is shown that among the three only the Tsallis 
entropy is stable. In Sec. 4, the axioms and the uniqueness theorem for the Tsallis entropy 
are discussed. Sec. 5 is devoted to concluding remarks. 

2. Composability 

Suppose the total system be divided into two independent subsystems, A and B. The 
joint probability of the total system is then factorized as p u (A, B) = p t {A) Pj(B), where 

Pi(A) is the probability of finding the subsystem, A, in its z'th state and so on. The 

concept of "composability" is stated as follows [1]. Entropy of the total system, 
S(A, B) = S[p(A, B)], is expressed in terms of entropies of the subsystems, S(A) and 
S(B). That is, 
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S(A,B) = f(S(A),S(B)), (1) 
where /is a symmetric bivariate function 

f(x,y) = f(y,x), (2) 

This function is referred to as the "composability function". The entropies frequently 
encountered in the literature satisfy the composability condition in eq. (1). The most 
celebrated one is the Boltzmann-Gibbs-Shannon entropy 

w 

S[p] = -£ PMPn (3) 

where W is the total number of microscopically accessible states at a given scale. This 
quantity is additive, i.e., 

S(A,B) = S(A) + S(B), (4) 

for independent A and B. Another additive entropy is the Renyi entropy [2], which is 
given by 



w 

(5) 



1 

S ( q R> [p] = — hi£(p ; r (q>0). 
1 Q i=i 
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The Tsallis entropy [3] 



si T) [pl 



l-q 



(<7>0) 



(6) 



is also composable, since it satisfies 



(A, B) = (A) + S< T) (B) + (l-q)S^(A)Sl"(B), 



(T)s A\ COT, 



(7) 



which is referred to as pseudoadditivity. The last example we mention here is the 
normalized Tsallis entropy [4,5] 



(q>0), 



(8) 



which also yields pseudoadditivity similar to eq. (7) 



(A, B) = (A) + (B) + (q-l) S ( q m > (A) S^' > (B) 



(9) 



We notice two important points. Firstly, the Renyi entropy, the Tsallis entropy and the 
normalized Tsallis entropy all converge to the Boltzmann-Gibbs-Shannon entropy in the 
limit q—tl. Secondly, the Renyi entropy and the normalized Tsallis entropy are concave 
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only for q e (0, 1), whereas the Tsallis entropy is always concave for any positive values 
of q. 

Now, what is remarkable here is that the above entropies have a common feature, 
"sum plus product": 



where r(q) - for the Renyi entropy as well as the Boltzmann-Gibbs-Shannon entropy, 
t(q) — 1 - q for the Tsallis entropy and t(q) - q - 1 for the normalized Tsallis entropy. 
In what follows, we shall see how this sum-plus-product structure is uniquely prescribed 
by the zeroth law of thermodynamics [6]. 

Let X be an additive quantity of the system. It may be the internal energy, the volume 
or the number of particles, for example. Its total amount is given by 



(It should be noticed that additivity of the internal energy is not obvious, in general. This 
makes the definition of temperature highly nontrivial for nonextensive systems like 
systems with long-range interactions. For the discussion about relaxing additivity of the 
internal energy, see Ref. [7].) Then, the stationary state of the total system may be 
defined by maximization of the total entropy with fixing X(A, B). That is, 



f(x, y) = x + y + f(q)xy, 



(10) 



X(A, B) = X(A) + X(B). 



(ID 



SS(A, B) = 8f(S(A),S(B)) = 0, 



(12) 
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SX(A) = -SX(B). 



(13) 



From these, it follows that 



df(S(A),S(B)) dS(A) = df(S(A),S(B)) dS(B) 
dS(A) dX(A) dS(B) dX(B)' 



The zeroth law of thermodynamics requires this equation to be separated as 

J(A) = J(B), (15) 

where J is temperature, pressure or chemical potential if X is the internal energy, the 
volume or the number of particles, respectively. Separation is realized if the derivatives 
of the composability function satisfy 



aS{A) 



df(S lt ) /» S , (B)) = k ( S ^> S ^ S(S(B)), (17) 

ob(B) 



where k, g and h are certain functions. It is not difficult to see that k is a symmetric 
function, i.e., k(x, y) = k(y, x), due to eq. (2). 
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We point out that, to establish eq. (15) with the correct physical dimensionality of J, k 
may be dimensionless. It is very much likely that in most of ordinary physical systems k 
is independent of the entropies of the subsystems. (A general case including an arbitrary 
symmetric function, k, can be found in Ref. [6].) Henceforth, k is taken to be unity. 
Accordingly, eqs. (16) and (17) become 



S(iTO4(Sm (18) 



dS(A) 



aS(B) 



respectively. 

Now, we impose the integrability condition 



d 2 f(S(A),S(B)) = d 2 f(S(A),S(B)) 
dS(A)dS(B) dS(B)dS(A) 



Then, from eqs. (18) and (19), we obtain 



1 dh(S(A)) 1 dh(S(B)) 



g(S(A)) dS(A) g(S(B)) dS(B) 



(20) 



= T, (21) 



where % is a separation constant. Using this equation in eqs. (18) and (19), we have 
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df(S(A)S(B)) 1 dhJS(AJ) 

o; t ^ W , JW; = i — T v hJS(B)), (22) 
35(A) t <*S(A) 



a/ t (5(A),5(B)) = l ^(5(5)) 
35(5) t T dS(B) 

It turns out that the case x = does not influence the final result (see below). Eqs. (22) 
and (23) are immediately integrated to yield 



f r (S(A), S(B)) = -h T (S(A))h T (S(B)) + c, (24) 
T 



where c is an integration constant. This is the form of the composability function 
compatible with the zeroth law of thermodynamics. 

Let us further determine the function, h t . For this purpose, suppose both A and B be in 

the completely ordered states. In this case, 5(A) = 5(5) = = 5(A, B) . Then, eq. (24) 
gives 



c = --h 2 T (0). (25) 



Without losing generality, h t (0) can be set equal to unity 



M0) = 1. (26) 
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Next, suppose only B be in the completely ordered state. Then, S(A, B)-S(A), and 
accordingly eq. (24) combined with eqs. (25) and (26) leads to 



K(S) = ts + i. 



(27) 



Substituting this into eq. (24), we find 



S(A, B) = f(S(A), S(B)) = 5(A) + S(B) + tS(A)S(B). 



(28) 



This precisely reproduces pseudoadditivity in eqs. (7) and (9). 

Thus, we have shown that validity of the zeroth law of thermodynamics puts a 
stringent constraint on the composition law for entropy and the resulting law is 
pseudoadditivity, i.e., the sum-plus -product structure. 

3. Stability 

In the previous section, we have seen how the Renyi entropy, the Tsallis entropy and 
the normalized Tsallis entropy (and the Boltzmann-Gibbs-Shannon entropy as the 
limiting case) are compatible with composability and existence of the stationary state, i.e., 
the generalized-maximum-entropy state defined consistently with the zeroth law of 
thermodynamics. In this section, we examine if these generalized entropies can be 
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physically observable quantities. For this purpose, we discuss the concept of stability 
introduced in Ref. [8]. 

Consider a quantity C - C[p] with its maximum value, C max . What is experimentally 

observed is usually not C but the distribution {p i } i :=lj2 ,...,w Repeating measurement of a 
certain physical random variable (e.g., energy), one obtains a set of distributions, which 
do not exactly coincide with each other. If C is a physically meaningful quantity, it 
should not drastically change under small change of the distribution. Otherwise, its values 
cannot be experimentally reproducible with reliability. Mathematically, this may be 
expressed as follows [8]: 



(Ve>0) {3d >0) 



p-p'W^d 



C[p]-C[p'] 



< £ 



(29) 



for arbitrarily large values of W, where HAH, stands for the l l norm of A: 
w I A, I. 

Let us examine stability of the generalized entropies in eqs. (5), (6) and (8). Note that 
all those entropies take their maximum values for the equiprobability p t =l/W 

(1 = 1,2,-, W): S^ = lnW, S^=\n q W, = -In, W~\ where In q x stands for 

the ^-logarithmic function defined by In x =(x l ~ q - 1) / (1 - q) ( x> 0) that tends to the 
ordinary logarithmic function, In x, in the limit q—>l. 

The following specific deformations of the distribution should be considered here: 
(i) 0<q<l; 
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Pi = S n , p'r 



1- 



8 W " 

2 W-ly 



Pi + 



8 1 
2 W-l' 



(30) 



(ii) q>U 



1 



v 



(31) 



These deformations are of physical importance in view of relaxation processes. In both 
cases, the I 1 norms are calculated to be II p — p' \\ x = 8. It is immediate to evaluate the 

changes of the Renyi entropy, the Tsallis entropy and the normalized Tsallis entropy 
under these deformations. The results in the thermodynamic limit, W — > °°, are given as 
follows: 

(i) 0<q<l; 



q, max 



1 



In 



Ml 



+ 



(w-i) 1 " 9 



->i 



lnW 

(W_>oo), 



(32) 



Sl T) [p]-S^[p'] 



AT) 
q, max 





q 




+ 




I V 







(w-\y- q -\ 



-» 



W l ~ q - 1 

(W->°o), 



(33) 
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f 



a (.NT) 
^ q, max 



1- 

V 2y 



1 

<5Y CSV 



+ 



(W-l) 



l-q 



-> 1 



(VK^oo). 



(34) 



(ii) q>U 



s™[p]-s?w 



g, max 



ln(W-l)-— In 
1-3 



<5Y 



+ 



.2; ^ 2; 



InW 



(W-l) 



l-q 



-> 1 



(35) 



( r ) r 



^, max 



2 J 



l 2j 



(W-l) 



1-9 



(W_>oo), 



(36) 



-(W-l) q ~ l + 



c (NT) 
" <j, max 



v2; 


9 

+ 


( 

1- 

V 


<5^ 

2; 


9 

(W-l) 1 " 9 


1- 


-w 


9-1 







->1. 



(VK^oo). 



(37) 
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It should be noticed that although the three generalized entropies are related to each other, 
for example, as S q T ) = (1 - q)~ l {exp[(l - q)S q R) ] - 1} = S q NT) /[1 + (q - l)S q m ], their 

stability properties are thus different. The Renyi entropy and the normalized Tsallis 
entropy do not satisfy the stability condition and therefore cannot be used for 
generalizing statistical mechanics. These entropies with < q < 1 overestimate a large 
number of occupied states even if their overall probability is irrelevantly small and these 
with q > 1 simply overestimate a high peak of probability. Accordingly, the Renyi entropy 
and the normalized Tsallis entropy cannot describe relaxation processes in a physically 
meaningful manner. (We also mention that although instability of the Renyi entropy has 
already been shown in Ref. [8] we have included it here for the sake of completeness of 
our discussion.) Among the three, only the Tsallis entropy satisfies the stability condition 
in eq. (29) under the above-mentioned specific (but important) deformations if 8 is taken 
to be 8<l£ llq . 

In what follows, we shall present a proof that the Tsallis entropy is in fact stable under 
an arbitrary deformation of the probability distribution. 

3. Stability of Tsallis entropy 

Let us consider 
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A,[/>;f) = £ 



(38) 



where t is a positive parameter and (x) + =max{x, 0}. It is possible to show that the 
following basic inequalities hold: 



A\p;t)-A\p;t)\<\\p-PK 



(39) 



A q [p;t)-A q [p';t) \< 



W 

-— (Vf>ln W). 
e q (0 



(40) 



A point is that the Tsallis entropy can be expressed in terms of A q [p, t) through the 
following integral representation [9] : 



Sl T) [p] = -l + q\ dt(l-A q [p;tj), 



(41) 



where ? ma X = 00 if < ^ < 1 and ? max = 1 / (q - 1) if q > 1. 



Using eq. (41), we find 



a+ln „ W 



S ( q T) [p]-S ( q T) [p']\<q J" dt\A q [p;t)-A q [p';t) 



+ q J dt\ A q [p;t)-A q [p';t) \, 



(42) 
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where a is a positive constant satisfying -\n q W < < a < r max . From eq. (39), the first 
term on the right-hand side of eq. (42) is shown to satisfy 



a+ln„ W 



q \ dt\A q [p;t)-A q [p';t)\<q\\p-p'WAa + \n q W), 



(43) 



whereas from eq. (40) the second term is found to be 



- max 

q \ dt\ A q [p;t)-A q [p';t)\< 



W 



a+ln„ W 



\e q {a + \n q W)] q 



(44) 



These enable us to rewrite eq. (42) as follows: 



S { q T) [p] ~ S q T) [p']\<q\\p-p'\\ l (a + \n q W) + 



W 



[e q (a + \n q W)\- 



(45) 



Evaluating the right-hand side of this inequality, one finds that it takes its minimum value 
when a = -\n q W +\n q (W/ II p — p'W^. Finally, replacing the right-hand side by its 



minimum value and recalling S q T) m . iX = In W, eq. (45) turns out to yield 



Sl T) [p]-Sl T) [p'] 



q, max 



<(HP- J P'II 1 ) 9 



f Xj \-q\* 9 \\p-p\\^ 



V 



ln ? W 



J 
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— > < 



(Wp-p'W.y (0<q<l) 
qWp-p'W, (q>l) 



(W->oo). 



(46) 



Therefore, taking 8 such that II p - p' ll x < 5 < e 9 for < q < 1 and II p - p' \\ r < 8 < £ I q 
for q>l, the stability condition in eq. (29) is found to be fulfilled by the Tsallis entropy. 

The above discussion is valid for \f q > 0, and accordingly stability of the Boltzmann- 
Gibbs-Shannon entropy [8] (recovered in the limit <^ — > 1) is also reestablished as a by- 



product. 



4. Axioms and uniqueness theorem 

It is well known that the Boltzmann-Gibbs-Shannon entropy is uniquely identified by 
a set of axioms termed the Shannon-Khinchin axioms [10,11]. In this section, we briefly 
mention that the Tsallis entropy has a similar axiomatic foundation. More detailed 
discussions can be found in Refs. [12,13]. 

The set of axioms presented in Ref. [13] is the following: [I] S q (p v p 2 ,---, p w ) is 

continuous with respect to all its arguments and takes its maximum for the 
equiprobability distribution Pi =\IW (i = 1, 2, W), [II] S q (A, B) = S q (A) + S q (B I A) 

+(l-q)S q (A)S q (B\A) for a bipartite system, (A, B), and [III] 

s q (Pv Pi,--;Pw> Pw + i = °) = s q (Pv Pi> •" Pw)- 11 can be shown [13] that the quantity S q 
satisfying [I] - [III] is, up to a multiplicative constant, uniquely given by 
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s q (Pi,P2>-, Pw) = s l T) [p] = r~ 

1 — I 



1=1 



(47) 



which is precisely the Tsallis in eq. (6). By comparison of this set of axioms with that of 
Khinchin [11], the one and only difference is seen to be in [II]. Khinchin's second axiom 
is recovered in the limit g— >1. 5^(5 I A) in [II] is the nonadditive conditional entropy 

defined by 



S(B\A)=<S(B\A i )> 



(A) 

q ' 



(48) 



provided that S q (B I A,.) is calculated from of the conditional probability of B with A 
found in its z'th state, p ij (B\ A) = p ij (A, B) / p^A) with the marginal probability 
Pi(A)-^. Pij(A, B). The symbol <Q> ( f > stands for the normalized ^-expectation 
value [14] defined by 



<G>Mfi ( iJ(A) , 



(49) 



where 



(50) 
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is the escort distribution [15] associated with p t (A). 

To avoid confusion, here we emphasize the following point. At the level of eq. (48), 
S (B I A ( ) is not specified yet. As a functional, this quantity should satisfy the axioms [I]- 

[III] for the conditional probability. After having reached at the result in eq. (47), overall 
consistency has to be ascertained. This can be done, for example, as follows. Calculate eq. 
(47) for the joint probability, Pij (A, B) = Pi (A) Pij (B I A), to obtain 

S { q T) (A, B) = ^ jL iPi( A ) Pu( B I A )V ~ 1 j 
= £[p i (A)]*Sf ) (BIA I .) + Sf ) (A) 
= 5< r) (5IA)£ [ Pi (A)r + S ( q T \A) 

i 

= Sl T) (A) + Sl T) (B I A) + (1 - q) Sl T) (A) (B I A), (51) 



which is consistent with [II]. Also, notice that there exists the following correspondence 
relation between the Bayes law and the generalized composition law in [II]: 



Pij (A, B) = Pi (A) Pij (B I A) = Pj (B) Pij (A I B) 



S q (A, B) = S q (A) + S q (B I A) + (1 - q)S q (A)S q (B I A) 
= S q (B) + S q (A I B) + (1 - ^(5) 5 9 (A I 5), 



(52) 
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which naturally generalizes that of the Boltzmann-Gibbs-Shannon entropy realized in the 
limit q -» 1: Pij (A, B) = p. {A) Pij (B I A) = Pj (B) p. .(A I B) <-> 5(A, 5) = S(A) + S(B I A) 

- S(B) + S(A I B). In [II], we clearly see the sum-plus-product structure discussed in Sec. 
2. It is of importance to notice that this generalized composition law is more general than 
pseudoadditivity since it holds even when there is correlation between A and B [16]. 

Finally, we wish to point out that the set of axioms for the Tsallis entropy presented in 
this section may provide a basis for nonadditive information theory (and its variants [17]). 
Nowadays it has further been generalized to quantum theory. In particular, resulting 
nonadditive quantum information theory has recently been discussed in the context of the 
problems of quantum entanglement, extensively [18-25]. 

5. Concluding remarks 

We have discussed the distinguished properties of the Tsallis entropy from three 
viewpoints: the principles of thermodynamics, the concept of stability and the axiomatic 
foundation. In particular, it is emphasized that the Tsallis entropy can provide a basis for 
generalizing statistical mechanics, whereas the Renyi entropy and the normalized Tsallis 
entropy are ruled out for such a purpose. Generalized statistical mechanics based on the 
Tsallis entropy, termed nonextensive statistical mechanics, is now under vital 
investigation. It has successfully been applied to a variety of complex systems. Its 
statistical and dynamical foundations have also been well developed. The references 
relevant to these developments can be found at URL [26]. 
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